PiMS: A Pre-ML Labelling Tool

Machine Learning (ML) techniques in clinical decision support systems are scarce due to the limited availability of clinically validated and labelled training data sets. We present a framework to (1) enable quality controls at data submission toward ML appropriate data, (2) provide in-situ algorithm assessments, and (3) prepare dataframes for ML training and robust stochastic analysis. We developed and evaluated PiMS (Pandemic Intervention and Monitoring Systems): a remote monitoring solution for patients that are Covid-positive. The system was trialled at two hospitals in Melbourne, Australia (Alfred Health and Monash Health) involving 109 patients and 15 clinicians.

PrISM-Tracker: A Framework for Multimodal Procedure Tracking UsingWearable Sensors and State Transition Information with User-Driven Handling of Errors and Uncertainty

A user often needs training and guidance while performing several daily life procedures, e.g., cooking, setting up a new appliance, or doing a COVID test. Watch-based human activity recognition (HAR) can track users' actions during these procedures. However, out of the box, state-of-the-art HAR struggles from noisy data and less-expressive actions that are often part of daily life tasks. This paper proposes PrISM-Tracker, a procedure-tracking framework that augments existing HAR models with (1) graph-based procedure representation and (2) a user-interaction module to handle model uncertainty. Specifically, PrISM-Tracker extends a Viterbi algorithm to update state probabilities based on time-series HAR outputs by leveraging the graph representation that embeds time information as prior. Moreover, the model identifies moments or classes of uncertainty and asks the user for guidance to improve tracking accuracy. We tested PrISM-Tracker in two procedures: latte-making in an engineering lab study and wound care for skin cancer patients at a clinic. The results showed the effectiveness of the proposed algorithm utilizing transition graphs in tracking steps and the efficacy of using simulated human input to enhance performance. This work is the first step toward human-in-the-loop intelligent systems for guiding users while performing new and complicated procedural tasks. © 2023 Owner/Author.

SanitizerBot: How Human-in-the-Loop Social Robots Can Playfully Support Humans

This paper evaluates a robot that distributed hand-sanitizer over an eight month period (October 2020-June 2021) in public places on the Oregon State University campus. During COVID times, many robots have been deployed in public places as social distancing enforcers, food delivery robots, UV-sanitation robots and more, but few studies have assessed the social situations of these robots. Using the context of robot distributing hand sanitizer, this work explores the benefits that social robots may provide to encouraging healthy human activities, as well as ways in which street-performance inspired approaches and a bit of humor might improve the quality and experience of functional human-robot interactions. After gaining human-in-the-loop deployment experience with a customized interface to enable both planned and improvized responses to human bystanders, we run two sub-studies. In the first, we compare the performance of the robot (moving or still) relative to a traditional hand sanitizer dispenser stick (N=2048, 3 week data collection period). In the second, we evaluate how varied utterance strategies further impact the interaction results (N=185, 2 week data collection period). The robot dramatically outperforms the stick dispenser across all tracked behavioral variables, cuing high levels of positive social engagement. This work finds the utterance design is more complex socially, and offer insights to future robot designers about how to integrate helpful and playful speech into service robot interactions. Finally, across both sub-studies, the work shows that people in groups are more likely to engage with the robot and each other, as well as sanitize their hands. © 2022 IEEE.

Post-pandemic healthcare for COVID-19 vaccine: Tissue-aware diagnosis of cervical lymphadenopathy via multi-modal ultrasound semantic segmentation.

With the widespread deployment of COVID-19 vaccines all around the world, billions of people have benefited from the vaccination and thereby avoiding infection. However, huge amount of clinical cases revealed diverse side effects of COVID-19 vaccines, among which cervical lymphadenopathy is one of the most frequent local reactions. Therefore, rapid detection of cervical lymph node (LN) is essential in terms of vaccine recipients' healthcare and avoidance of misdiagnosis in the post-pandemic era. This paper focuses on a novel deep learning-based framework for the rapid diagnosis of cervical lymphadenopathy towards COVID-19 vaccine recipients. Existing deep learning-based computer-aided diagnosis (CAD) methods for cervical LN enlargement mostly only depend on single modal images, e.g., grayscale ultrasound (US), color Doppler ultrasound, and CT, while failing to effectively integrate information from the multi-source medical images. Meanwhile, both the surrounding tissue objects of the cervical LNs and different regions inside the cervical LNs may imply valuable diagnostic knowledge which is pending for mining. In this paper, we propose an Tissue-Aware Cervical Lymph Node Diagnosis method (TACLND) via multi-modal ultrasound semantic segmentation. The method effectively integrates grayscale and color Doppler US images and realizes a pixel-level localization of different tissue objects, i.e., lymph, muscle, and blood vessels. With inter-tissue and intra-tissue attention mechanisms applied, our proposed method can enhance the implicit tissue-level diagnostic knowledge in both spatial and channel dimension, and realize diagnosis of cervical LN with normal, benign or malignant state. Extensive experiments conducted on our collected cervical LN US dataset demonstrate the effectiveness of our methods on both tissue detection and cervical lymphadenopathy diagnosis. Therefore, our proposed framework can guarantee efficient diagnosis for the vaccine recipients' cervical LN, and assist doctors to discriminate between COVID-related reactive lymphadenopathy and metastatic lymphadenopathy.

PiMS: A Pre-ML Labelling Tool

Machine Learning (ML) techniques in clinical decision support systems are scarce due to the limited availability of clinically validated and labelled training data sets. We present a framework to (1) enable quality controls at data submission toward ML appropriate data, (2) provide in-situ algorithm assessments, and (3) prepare dataframes for ML training and robust stochastic analysis. We developed and evaluated PiMS (Pandemic Intervention and Monitoring Systems): a remote monitoring solution for patients that are Covid-positive. The system was trialled at two hospitals in Melbourne, Australia (Alfred Health and Monash Health) involving 109 patients and 15 clinicians. © 2022 IEEE.

The Best of Both Worlds: Mixed Systems with ML and Humans in the Loop to Combat Fake Information

Dealing adequately with misinformation is one of the societal challenges of our times, since misinformation has been proven to be harmful for people, societies, and democracy. Improving Artificial Intelligence algorithms underlying information retrieval and recommendation systems is a path that must be encouraged;however, this is not the only path ahead and, above all, it can be combined with other approaches. In addition to the known limitations of Machine Learning model results, which cannot be guaranteed to be 100% accurate, ethical issues are raised when an algorithm acts as a censor of what information a human being can and cannot access. This paper discusses some recent initiatives during the COVID-19 pandemic to improve the quality of information delivered to users that have two characteristics in common: firstly, they are technically simple, hard coded, and do not involve any AI;secondly, they represent a preliminary step in a broader perspective that goes beyond technical improvements to promote critical thinking among those receiving the information. Although they can be seen as preliminary cases of how to deal with misinformation, they seem to be effective and they point towards more interdisciplinary solutions to the contemporary issue of misinformation, possibly bringing other developments to ethical, Human-Centered AI. © 2022, Springer Nature Switzerland AG.

A Remote, Human-in-the-Loop Evaluation of a Multiple-Drone Delivery Operation

Over time, advances in unmanned aircraft systems (UAS) have enabled a shift in the operational paradigm from one operator managing one aircraft to that of multiple operators working together to manage multiple aircraft. This shift has highlighted the need for effective human-autonomy teaming methods to maintain manageable workload levels for operators as well as high standards of system performance and safety. This paper presents a study aimed at evaluating whether automation can help operators manage workload during small UAS (sUAS) package delivery scenarios featuring contingency situations. These contingency situations, resulting from unplanned UAS Volume Reservations (UVRs), required flight path reroutes for multiple aircraft simultaneously. The study manipulated the number of aircraft affected by the UVRs and the level of automation support. The presence of terrain conflicts was also controlled within each scenario. Due to the COVID-19 pandemic, subjects were not able to gain direct access to the Ground Control System (GCS). Therefore, the study was conducted using a subject-surrogate paradigm that required subjects to relay commands through a verbal protocol from remote locations outside of the lab to a researcher surrogate who had direct control of the GCS interfaces at the lab location. Results show that the automated support condition was associated with faster reroute response times, more efficient reroute maneuvers, and significantly lower levels of perceived workload than the manual reroute condition. However, the automation support level did not significantly impact pilots’ ability to avoid the UVR successfully;pilots were overwhelmingly capable of avoiding the UVR in all conditions. The presence of terrain conflicts primarily impacted pilot performance by leading to multiple uploads per vehicle, which was not typically required when pilots only needed to maneuver laterally. Although subjects did not have direct control over the GCS, subjective ratings indicate that the displays under test provided them with sufficient information to manage their aircraft and promptly respond to the unplanned UVRs. Overall, the objective and subjective data strongly suggest that the verbal protocol and subject-surrogate paradigm were effective methods for collecting data remotely amid the COVID-19 pandemic. © 2022, American Institute of Aeronautics and Astronautics Inc, AIAA. All rights reserved.

Game-Theoretic Frameworks for Epidemic Spreading and Human Decision-Making: A Review.

This review presents and reviews various solved and open problems in developing, analyzing, and mitigating epidemic spreading processes under human decision-making. We provide a review of a range of epidemic models and explain the pros and cons of different epidemic models. We exhibit the art of coupling between epidemic models and decision models in the existing literature. More specifically, we provide answers to fundamental questions in human decision-making amid epidemics, including what interventions to take to combat the disease, who are decision-makers, and when and how to take interventions, and how to make interventions. Among many decision models, game-theoretic models have become increasingly crucial in modeling human responses or behavior amid epidemics in the last decade. In this review, we motivate the game-theoretic approach to human decision-making amid epidemics. This review provides an overview of the existing literature by developing a multi-dimensional taxonomy, which categorizes existing literature based on multiple dimensions, including (1) types of games, such as differential games, stochastic games, evolutionary games, and static games; (2) types of interventions, such as social distancing, vaccination, quarantine, and taking antidotes; (3) the types of decision-makers, such as individuals, adversaries, and central authorities at different hierarchical levels. A fine-grained dynamic game framework is proposed to capture the essence of game-theoretic decision-making amid epidemics. We showcase three representative frameworks with unique ways of integrating game-theoretic decision-making into the epidemic models from a vast body of literature. Each of the three frameworks has their unique way of modeling and analyzing and develops results from different angles. In the end, we identify several main open problems and research gaps left to be addressed and filled.

Active preference-based optimization for human-in-the-loop feature selection

In various classification problems characterized by a large number of features, feature selection (FS) is essential to guarantee generalization capabilities. The FS problem is often ill-posed due to significant correlations among features, which may lead to several different feature subsets with comparable scores in terms of classification performance. However, not all these subsets are equivalent from a domain-oriented point of view due to known relationships among features and their different acquisition costs in production to deploy the trained classifier. In this paper, we consider the potential benefits of including the domain expert’s preferences in the FS task, thus integrating both objective elements (e.g., classification accuracy) and subjective (often not quantifiable) considerations in the selection process. This goes in the direction of increasing the interpretability and the trustworthiness of the machine learning model, which is an often desired property in many application domains such as in medicine. The proposed method consists of an iterative procedure. At each iteration, the expert is asked to express a “human” preference on pairs of classifiers, each one trained from a different subset of features. The expressed preferences are used algorithmically to update a suitable surrogate function that mimics the latent subjective expert’s objective function, and then to propose a new classifier for testing and comparison. The proposed method has been tested on academic and experimental FS problems, and notably, on a COVID’19 patients record. The preliminary experimental results are promising, in that a parsimonious and accurate solution is obtained after a relatively short number of iterations.

Healing Hands: The Tactile Internet in Future Tele-Healthcare.

In the early 2020s, the coronavirus pandemic brought the notion of remotely connected care to the general population across the globe. Oftentimes, the timely provisioning of access to and the implementation of affordable care are drivers behind tele-healthcare initiatives. Tele-healthcare has already garnered significant momentum in research and implementations in the years preceding the worldwide challenge of 2020, supported by the emerging capabilities of communication networks. The Tactile Internet (TI) with human-in-the-loop is one of those developments, leading to the democratization of skills and expertise that will significantly impact the long-term developments of the provisioning of care. However, significant challenges remain that require today's communication networks to adapt to support the ultra-low latency required. The resulting latency challenge necessitates trans-disciplinary research efforts combining psychophysiological as well as technological solutions to achieve one millisecond and below round-trip times. The objective of this paper is to provide an overview of the benefits enabled by solving this network latency reduction challenge by employing state-of-the-art Time-Sensitive Networking (TSN) devices in a testbed, realizing the service differentiation required for the multi-modal human-machine interface. With completely new types of services and use cases resulting from the TI, we describe the potential impacts on remote surgery and remote rehabilitation as examples, with a focus on the future of tele-healthcare in rural settings.